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ABSTRACT 

We study the production and decay of a coupled system of mixed neutral MSSM Higgs 
bosons in exclusive double-diffractive processes at the LHC, including non-vanishing CP 
phases in the soft supersymmetry-breaking gaugino masses and third-generation trilinear 
squark couplings. The three neutral Higgs bosons are naturally nearly degenerate, for 
large values of tan /3, when the charged Higgs boson weighs around 150 GeV. Large mixing 
between all three neutral Higgs bosons is possible when CP is violated, a three-way mixing 
scenario which we also term tri-mixing. A resolution in the Higgs mass of ~ 1 GeV, 
which may be achievable using the missing-mass method, would allow one to distinguish 
nearly degenerate Higgs bosons by studying the production lineshape. Measurements of 
the polarizations of the tau leptons coming from the Higgs-boson decays could offer a direct 
and observable signal of CP violation in the Higgs sector. 



1 Introduction 



Direct search experiments at LEP established that, within the Standard Model, the single 
physical Higgs boson must weigh more than about 114 GeV [1], and the mass range favoured 
by indirect measurements is close to this lower limit [2] . Both the direct and indirect limits 
must be re-examined in non-minimal Higgs scenarios. One popular example is the minimal 
supersymmetric extension of the Standard Model (MSSM) [3], which predicts the existence 
of at least one light neutral Higgs boson weighing less than about 135 GeV. This may 
be joined by two other light neutral Higgs bosons if the charged Higgs bosons are 
also light. If CP is conserved, only the two CP-even neutral Higgs bosons h, H may mix 
with each other dynamically, through off-diagonal absorptive self-energy effects. Instead, 
if CP is violated, all the three neutral Higgs bosons, including the CP-odd neutral Higgs 
boson A, mix through CP-violating quantum effects [4-8] to mass eigenstates, #1,2,3, of 
indefinite CP. These CP-mixed Higgs states give rise to a coupled system whose resonant 
dynamics is properly described by a 3-by-3 propagator matrix [9,10]. One very interesting 
phenomenological feature of the CP-violating MSSM is that the lightest neutral Higgs 
boson could be considerably lighter than 114 GeV [11,12]. 

Inclusive experiments at the LHC will be able to discover the Standard Model Higgs 
boson, whatever its mass. They should also be able to discover the lightest neutral MSSM 
Higgs boson — at least if CP is conserved — and will be able to explore significant mass 
ranges for the heavier MSSM Higgs bosons. Supplementing inclusive searches, interest has 
recently been growing in the search for Higgs bosons in diffractive events at the LHC [13, 
14]. These may offer novel prospects for measuring the properties of light neutral Higgs 
bosons [15], and disentangling their CP properties [16]. 

The MSSM offers additional sources of CP violation beyond the single Kobayashi- 
Maskawa phase in the Standard Model. If the soft supersymmetry-breaking parameters 
m o, m i/2 and A are universal, two new physical CP-odd phases remain. These may, without 
loss of generality, be parametrized as one phase in the trilinear couplings A and one in the 
gaugino masses m^j. In addition to signatures of CP violation in sparticle production and 
decay at high-energy colliders [17-19], these phases may have observable radiative effects on 
the Higgs sector [4], on electric dipole moments [20-22], and in B decays [23,24]. One of the 
principal motivations for studying such models is the prospect of electroweak baryogenesis 
in the MSSM [25]. 

In this paper, we consider the prospects for studying light neutral MSSM Higgs bosons 
in diffractive events at the LHC, particularly in scenarios where CP is violated [4-8,26-30], 
and the three neutral Higgs bosons mix strongly. This work continues previous studies of the 
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masses, couplings, production and decays of the mixed-CP Higgs bosons #1,2,3, with a view 
to searches at LEP [11], the LHC [11,12,15,16,31-33], the ILC [34], a n+n~ collider [35] and 
a 77 collider [36-40]. As in our previous works [10,40], we include a complete treatment of 
loop-induced CP violation and three-way mixing, including off-diagonal absorptive effects 
in the resummed Higgs-boson propagator matrix [9]. 

Higgs-boson production in an exclusive diffractive collision p + p — * p + Hi + p, 
where the outgoing protons remain intact and scatter through small angles, offers a unique 
environment for investigating the MSSM Higgs sector, in particular when tan (3 is large and 
MP± e ~ 150 GeV. In such scenarios, all three neutral Hiffffs bosons have similar masses and 
there is strong three-way mixing. We call a scenario with these properties the three-way 
mixing or tri-mixing scenario of the CP-violating MSSM. 

Moreover, the production cross section of the Higgs boson can be much enhanced in 
the MSSM for large values of tan (3, as compared with the Standard Model. Furthermore, 
good Higgs-mass resolution of the order of 1 GeV may be achievable [13] by precise mea- 
surements of the momenta of the outgoing protons in detectors a long way downstream of 
the interaction point. This enables one to disentangle nearly degenerate Higgs bosons by 
examining the production lineshape of the coupled system of neutral Higgs bosons. 

The layout of this paper is as follows. Section 2 provides basic formulae for the 
luminosity of the exclusive double-diffractive process, based on [41,42]. Section 3 presents 
the formalism for the production, mixing and decay of a coupled system of CP-violating 
neutral MSSM Higgs bosons in diffraction, based on [10]. It also presents numerical example 
in a couple of CP- violating scenarios, considering two final states: bb and t + t~. Our 
conclusions are given in Section 4. 



2 Luminosity for the Exclusive Double Diffractive 
Process 

The effective luminosity for producing via double diffraction a system of invariant mass M 
and rapidity y can be written as [41] 

where the 'soft' survival factor S 2 is quite model- and process-dependent. Denoting the 
longitudinal momenta of two gluons that fuse into a system with invariant mass M by xipi 
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and X2P2, we have 

xi = —= e y , x 2 = -= e~ y , (2) 

where s = (pi + P2) 2 is the collider centre-of-mass energy squared, see Fig. [TJ 




Figure 1: The mechanism contributing to the exclusive double- diQr active process. 

Assuming J z = 0, a parity-even central system and negligibly small perpendicular 
momenta of the outgoing protons: p± <C 1 GeV, the expression for L becomes [41,42] 



7T 



(N 2 C - 1)6 Jau Q' T 



f g (xi, x 1 , Q T , /i ) f g (x 2 , x 2 , Qt, ^ ) 



n 2 



(3) 



where Nq = 3 and Q\ is the virtuality of the soft gluon needed for colour screening. 
The hard scale fi and the t-slope parameters are chosen as follows: \i = M/2 and b = 
4 GeV -2 . Formally, we also have introduced a non- vanishing cutoff Q m m ~ 1 GeV to 
avoid encountering the Landau pole, but the sensitivity of L to Q m i n turns out to be 
not large. Furthermore, if x' <C x (which is actually the case for the exclusive diffractive 
process [41,42]), the skewed or off-diagonal unintegrated gluon density f g (x, x', Qj>, H 2 ) niay 
take on the factorizable form: 



fg(X,X,Qrp,fi ) ~ Rgfg(X,Q T ,fi ) 



(4) 



where R g is a constant. The simplified form (0J) is estimated to have an accuracy of 10 to 
20%, where the function f g (x,Q^,fi 2 ) is defined by 



f g (x, Q T , n 



d 



dhiQ 2 T 



-y/T(Q T ,fl)xQ T 



T(Q T ,n)xg{x,Q 2 T 

dg(x,Ql) g{x,Ql) dS{Q T ,v) 



(5) 
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We note that f g (x, Q T , Li 2 ) consists of derivatives of the gluon distribution function g(x, Q\) 
and the Sudakov factor T(Qt,i~l) = e~ s ^ T '^\ Collecting all the factors, the effective 
luminosity can be rewritten as 



d 2 C 
dydM 2 



S 2 



ttR 2 



8 b JlnQ mi 



F 9 (x 1 ,x 2 ,Qt, lA dlnQ T 



(6) 



where the integrand functions F g (x\, x 2 , Qt, h) is defined by 



Fg(x 1 ,X2,Q T ,v) 



fg{x u Q 2 T ^ 2 )f g {x 2 ,Q 2 T ^ 2 



Q 



(7) 



T 



for the double- diffract ive process. 

For our numerical analysis, we also need to know the parton distribution functions 
(PDFs) for g(x, Q 2 T ) in f g (x, Q 2 T , fx 2 ). Specifically, we take the PDFs given by CTEQ6M [44] 
and MRST2004NNLO [45]. The function S(Qt,/^) that determines the Sudakov factor 
T(Qr,Ai) can be calculated as 



S(Qt,li) = — \ 

Z7T JC 



1 f^a s (k 2 ) 



2 p 



Fgfc) + E F q (k t 

q=u,d,s,c,u,d,s,c 



where 

T q ik t ) 



l-A(fet) 







zP gg (z)dz = -ll/2-61nA + 12A-9A 2 + 4A 3 -3A 4 /2 



1-A(fe t ) 



P qg (z)dz = 1/3 - A/2 + A 2 /2 + A 3 /3 



(9) 



with A = A(k t ) = k t /{n + k t ). 

Equations (HJ)-© provide a complete basis for our numerical evaluation of the effective 
luminosity for exclusive double-diffractive Higgs production. In particular, we find that the 
effective luminosity may conveniently be computed as follows: 



M 



d 2 C 
dydM 2 



4.0 x 10" 



& miD F g (xi,x 2 , Qt, h) dlnQ T 



GeV- 



S 2 



0.02/ \b GeV z 



2 / p x 4 

Kg 



1.2 



(10) 

We note that the luminosity is very sensitive to the choice of R g , in particular, but less to 
the hard scale \i (see also our discussion below). 

In Fig. El we show xg(x, Q T ) (upper left), the Sudakov factor T(Qt, li) (upper right), 
f g (x,Q T , /I 2 ) (lower left), and F g (xi, x 2 , Qt) (lower right), see (J7J), as functions of Qt or 
IuQt when the rapidity y = 0, M = 120 GeV, and i/i = 14 TeV. The solid (dashed) lines 
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Figure 2: The quantities xg(x,Q T ) (upper left), the Sudakov factor T(Qt, fi) (upper right) , 
fg( x , Qti A* 2 ) (lower left), and F g (x\, x 2 , Qt, f) (lower right), as functions of Qt or In Qt 
when the rapidity y = 0, M = 120 GeV, and \fs = 14 TeV. The solid (dashed) lines are 
the outputs of CTEQ6M (MRST2004NNLO). 

are outputs of CTEQ6M (MRST2004NNLO). We observe significant differences between 
the functions xg(x, Q T ), which results in a strong dependence on the parton distribution 
function used. Specifically, MRST2004 gives a non- vanishing and partly negative xg(x, Q T ) 
when Q T < 0.9 GeV, where f g (xi >2 , Q T , /i 2 ) becomes negative due to the rapidly decreas- 
ing xg(x,Q T ). On the other hand, CTEQ6M returns g(x,Q T ) = in this region. For 
0.9 GeV < Q T < 10 GeV, MRST2004 returns smaller values for F g (x 1 , x 2 , Qt, ft) than 
CTEQ6M does. When ln(Q T /GeV) > 3 (Q T > 20 GeV), the contribution from this large- 
Qt region to the luminosity is negligible, making the prediction insensitive to the choice of 
fi > 20 GeV. We note that the most significant contribution comes from the region around 
Qt = 1.3 GeV, as seen in the lower-right frame for F g (xx, x 2 , Qt, A*)- 

In fact, the calculations of g(x,Q T ) in both the CTEQ6 and MRST2004 parameter- 
izations are not reliable when Qt < 1 GeV. There are some phenomenological prescrip- 
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tions for making the low-Q^ behaviour more sensible [43]. For Higgs bosons with masses 
M ~ 120 GeV, however, the luminosity does not depend strongly on the variation between 
the prescriptions, and we take Q m m = 1 GeV to avoid unphysical effects associated with 
the inapplicability of the g(x, Qy) calculation for < 1 GeV. 

.2 
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Figure 3: The effective luminosity for the exclusive double- diffractive process for = 14 
TeV as a function of M (y = 0) and y (M = 120 GeV) in the upper (lower) frame, 
respectively. Solid (dashed) lines are obtained using CTEQ6M (MRST2004NNLO), taking 
Qmm — 1-0 GeV in each case. Predictions using other PDFs lie between the above two 
extreme cases, according to the analysis in [42]. 

In Fig. El we show the effective luminosity for the exclusive double-diffractive process 
at the LHC as a function of M for y = (upper frame) and as a function of y for M = 120 
GeV (lower). The solid (black) lines are obtained using CTEQ6M and the dashed (red) 
lines by MRST2004NNLO. We take Q min = 1.0 GeV, S 2 = 0.02, and R g = 1.2 [41], 
see (II Oj) . We note large differences between the two predictions over large regions of M and 
y, which should be regarded as two extreme cases. For example, at (y, M) = (0, 120 GeV), 
the CTEQ6M prediction is about twice as large as that of MRST2004NNLO, and we 
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find that the prediction of CTEQ6M has a stronger dependence on y than does that of 
MRST2004NNLO. We remark that the predictions using other PDFs lie between the above 
two extreme cases, as discussed in [42]. 



3 The Process pp^p + Hi+p^p^r [f(o)f{o)\ + p 

The helicity amplitude for the process gt{M)g\{^2) — ► Hi — > f(a)f(a) in the double- 
diffractive production of Higgs bosons Hi is given by 

M(aa; A,A 2 ) = <*>/ , (H) 

where the amplitude (a) f is defined as 

(*)f = E S?(V§) D^s) {o(3 f g s Hjff - ig P Hjff ) . (12) 

i,j=l,2,3 

For the jf = + process with pj_ that we consider, the pseudoscalar form factor of 
the g-g-Hi vertex does not contribute, making the helicity amplitude independent of the 
helicities of gluons. For the definitions of the couplings, the threshold corrections that are 
enhanced for large values of tan (5 for / = b , r, and the full 3x3 propagator matrix Dij(s), 
we refer to [10,46]. 

Similarly as in [10], one can define the parton-level cross section as: 

H - 2{Nl-l)ML( 9^m f ^s \ 2 f 

v c ; 512tts ^ 8nvM w ) 1 K J 

where the enhancement factor 2(Nq — 1) for the exclusive process and color factors Nf : 
Ni — 1 and N q = 3 have been included. The coefficients C( are given in terms of the 
amplitudes (a) f 

C{ = \(\(+)f\ 2 + \(-)f\ 2 ), C{ = l(\(+)f\ 2 -\(-) f \ 2 ), 

C{ = -M(+)f(-r f ), C{ = Zm({+) ,{-)}). (14) 

For our numerical results, we make the following choices of parameters: 
tan/? = 50, MP°i c = 155 GeV, 

Mq 3 = M &3 = M bi = M u = = Msusy = 0.5 TeV, 

M = 0.5 TeV, \A t ,b, T \ = 1 TeV, |M 2 | = \M±\ = 0.3 TeV, |M 3 | = 1 TeV, 

$ M = 0°, $ A = $ At = $ Ab = $ At = 90°, $ 1 = $ 2 = 0°, (15) 
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and we consider two values for the phase of the gluino mass parameter M3: $3 = —10° , —90°. 
For $3 = —10°, CPsuperH [46] yields for the masses and widths of the neutral Higgs bosons: 



M Hl = 120.2 GeV, M H2 = 121.4 GeV, M Hz = 124.5 GeV, 

V Bl = 1.19 GeV, T H2 = 3.42 GeV, rV 3 = 3.20 GeV, (16) 

and for $ 3 = -90°: 

M Hl = 118.4 GeV, M H2 = 119.0 GeV, M Ha = 122.5 GeV, 

T Hl = 3.91 GeV, T H2 = 6.02 GeV, r^ 3 = 6.34 GeV, (17) 

respectively. In the above MSSM scenario, which has originally been introduced in [10,40], 
all the three neutral CP-violating Higgs bosons are nearly degenerate, with masses of 
~ 120 GeV, and mix strongly through off-diagonal absorptive self-energy effects. We call 
a scenario with these properties, which can only be realized in a CP-violating MSSM, the 
three-way mixing or tri-mixing scenario. 

For / = b, the polarization of the b quark and the b anti-quark cannot be measured, 
and the only observable is the total cross section, which is given by 



dydM 2 1 dydM 2 ' v ' 

where we include the perturbative QCD correction K = 1 + ^^-(n 2 + 11/2) [47]. The 
factor 4 comes from the sum over the b- and 6-quark polarizations [15]. 

Figure 0] displays two numerical examples of three-way mixing scenarios with $3 = 
—90° (solid lines) and $3 = —10° (dashed lines). In this case, we have taken CTEQ6M and 
Qmin = 1 GeV. The differential cross section becomes as large as ~ 13 fb for $ 3 = —90°, 
in which case it exhibits a single peak located between the pole masses, that are indicated 
by the solid vertical lines. This peak is broader than the expected missing-mass resolution 
5M ~ 1 GeV. On the other hand, twin peaks are discernible when $3 = —10°, thanks to 
the expected good resolution in the Higgs mass: 5M ~ 1 GeV. The twin peaks appear 
close to (but not at) the outer pair of pole masses, indicated by the vertical dashed lines, 
and the cross section actually exhibits a dip at the second pole mass. These examples 
demonstrate that the unique sensitivity to CP-conserving observables provided by the good 
mass resolution in double-diffraction events would in turn provide sensitivity to the CP- 
violating phase $3. 

For / = r, we have four observables which can be constructed from aj with % — 1 — 4. 
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Figure 4: The hadronic level cross section M 2 d (y = 0) when the produced Higgs 
bosons decay into b quarks, calculated using CTEQ6M PDFs. Tri-mixing scenarios have 
been taken with $3 = —90° (solid lines) and $ 3 = —10° (dotted lines). The vertical lines 
indicate the three Higgs-boson pole-mass positions. 



In particular, to analyze the signatures of CP violation in the production of longitudinally- 
polarized r leptons, we define 



Atfcp = o-(pp -> pHip ; Hi -> t^t r ) - a(pp -> pHip ; H { -> t£t l 

We then have the total and CP-violating cross sections given by 

d 2 C 



(19) 



M 



M' 



dydM 2 
d 2 Aa r CP 



dydM 2 
d 2 C 



AKo-lM 2 - 

2 dydM 2 



dydM 2 

Finally, we define the CP- violating asymmetry oqp as 



M' 



,d 2 Ac 



a CP — 



dydM 2 



M 2 d2< > 



dydM 2 



(20) 



(21) 
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Figure El displays numerical examples in tri-mixing scenarios with $3 = —90° (solid 
lines) and $3 = —10° (dotted lines). We have again taken CTEQ6M and Qmin = 1 GeV, 
and find cross sections as large as ~ 1.5 — 3 fb. The peaks and dips in the total cross 
section are located relative to the pole masses in the same way as for the bb final state, 
again offering sensitivity to the CP- violating phase. Moreover, comparing Figs.lUandEl we 
see that the relative sizes of the peaks are different for the two values of $3, thus providing 
more sensitivity to this CP-violating parameter. 
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Figure 5: The hadron-level CP-conserving and CP-violating cross sections when Higgs 
bosons decay into r leptons: M 2 q q^/ 2 (left) an d M 2 (right), calculated using 

CTEQ6M PDFs. We have considered tri-mixing scenarios with $3 = —90° (solid lines) 
and $3 = —10° (dotted lines). The vertical lines indicate the three Higgs-boson pole-mass 
positions. 

The CP-violating cross-section difference observable in the r + r~ final state is shown 
in the right panel of Fig. El In the case $3 = —90°, the cross-section difference always has 
the same sign and is maximized between the pole masses, whereas in the case $3 = —10° it 
is generally smaller and exhibits two sign changes. The expected missing-mass resolution 
5M ~ 1 GeV should be sufficient to resolve some of these structures. We expect that a CP 
asymmetry a T cv larger than 10 % may be detected with an integrated luminosity Z 100 fb -1 . 
As we see in Fig. El the CP- violating observable a T c -p attains values considerably larger than 
10% in both the scenarios studied. Whereas the individual cross sections depend on the 
PDFs used, the CP asymmetry shown in Fig. El is insensitive to this choice. 
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Figure 6: The CP-violating asymmetry a£p observable in tri-mixing scenarios when Higgs 
bosons decay into r leptons. The line styles are the same as in Fig. [3J 



4 Conclusions 



We have extended our previous studies of CP-violating MSSM Higgs scenarios with large 
tri-mixing [10, 40] to diffractive production at the LHC. The production cross sections 
are large compared to those in the Standard Model, and the good Higgs-mass resolution 
obtainable via the missing-mass method should enable one to disentangle the different 
adjacent resonant peaks. Although it is difficult to construct CP- violating observables in 
Hi — > bb decays without tagging the final protons [16] or analyzing the 6-quark decay 
products, observations of r ± helicities would permit a CP asymmetry to be measured in 
Hi — ► t + t~ decays. This information may also be used to further resolve the underlying 
resonant dynamics of a strongly mixed Higgs-boson system, thereby offering a sensitive 
window into CP violation and new physics due to an extended Higgs sector, such as the 
MSSM Higgs sector. 
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This example shows that exclusive double diffraction may offer unique possibilities for 
exploring Higgs physics in ways that would be difficult or even impossible in inclusive Higgs 
production. In particular, we have shown that exclusive double diffraction constitutes an 
efficient CP and lineshape analyzer of the resonant Higgs-boson dynamics in multi-Higgs 
models. In the specific case of CP-violating MSSM Higgs physics discussed here, which is 
potentially of great importance for electroweak baryogenesis, diffractive production may be 
the most promising probe at the LHC. 
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